Method and apparatus for tracking an at least partially occluded object, vehicle and computer-program product thereof

ABSTRACT

A method for tracking an at least partially occluded object. The method includes recognizing a non-occluded portion of the at least partially occluded object in an input image; generating a simulated image of the at least partially occluded object based on features of the non-occluded portion extracted from the input image; determining first coordinates of the at least partially occluded object in a first coordinate system; and converting the first coordinates of the at least partially occluded object in the first coordinate system into second coordinates in a second coordinate system defined in a display apparatus.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Chinese Patent Application No.201810041758.8, filed Jan. 16, 2018, the contents of which areincorporated by reference in the entirety.

TECHNICAL FIELD

The present invention relates to smart object-tracking technology, moreparticularly, to a method for tracking an at least partially occludedobject, an apparatus for tracking an at least partially occluded object,a vehicle, and a computer-program product.

BACKGROUND

In recent years, objection recognition devices for vehicular use havebeen developed. The objection recognition devices can be used torecognize, e.g., a moving object such as a vehicle traveling ahead ofthe user's vehicle, or a pedestrian. Thus, the objection recognitiondevices can detect a dangerous situation in its early stage, and notifythe driver avoid the danger.

SUMMARY

In one aspect, the present invention provides a method for tracking anat least partially occluded object, comprising recognizing anon-occluded portion of the at least partially occluded object in aninput image; generating a simulated image of the at least partiallyoccluded object based on features of the non-occluded portion extractedfrom the input image; determining first coordinates of the at leastpartially occluded object in a first coordinate system; and convertingthe first coordinates of the at least partially occluded object in thefirst coordinate system into second coordinates in a second coordinatesystem defined in a display apparatus.

Optionally, generating the simulated image of the at least partiallyoccluded object comprises classifying the non-occluded portion usingclassifiers trained using extracted features of portions of a pluralityof training objects; wherein the simulated image is an image of one ofthe plurality of training objects, a portion of which comprisingextracted features closest matching the features of the non-occludedportion extracted from the input image.

Optionally, converting the first coordinates of the at least partiallyoccluded object in the first coordinate system into the secondcoordinates in the second coordinate system defined in the displayapparatus comprises converting the first coordinates of the at leastpartially occluded object in the first coordinate system into thirdcoordinates in a third coordinate system; and converting the thirdcoordinates in the third coordinate system into the second coordinatesin the second coordinate system defined in the display apparatus;wherein the first coordinate system is a three-dimensional coordinatesystem; and the third coordinate system is a two-dimensional coordinatesystem on a plane substantially parallel to an imaging plane of thedisplay apparatus.

Optionally, converting the first coordinates of the at least partiallyoccluded object in the first coordinate system into the thirdcoordinates in the third coordinate system is performed according toEquation (1) and Equation (2) when the imaging plane of the displayapparatus is substantially parallel to a coordinate plane of the firstcoordinate system:

$\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (x1, y1) stands forcoordinate differences between the first coordinates and a referencecoordinate of a reference point in the first coordinate system; f standsfor a length of an orthographic projection of a distance between theimaging plane and the reference point on a coordinate axis in the firstcoordinate system perpendicular to the imaging plane; and l stands for alength of an orthographic projection of a distance between the at leastpartially occluded object and the reference point on the coordinate axisin the first coordinate system perpendicular to the imaging plane.

Optionally, the reference point is a position of a view zone of a viewerin a vehicle.

Optionally, converting the third coordinates in the third coordinatesystem into the second coordinates in the second coordinate systemdefined in the display apparatus is performed according to Equation (3)and Equation (4):

$\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (u, v) stands for thesecond coordinates; (u₀, v₀) stands for a reference coordinate of areference point in the third coordinate system; dx stands for a realdistance representing a first pixel pitch along a first direction of thedisplay apparatus; dy stands for a real distance representing a secondpixel pitch along a second direction of the display apparatus.

Optionally, the reference point is a position of an orthographicprojection of a view zone of a viewer in a vehicle on the imaging plane.

Optionally, the non-occluded portion of the at least partially occludedobject is recognized in at least one frame from a first frame of imageto an (N−1)-th frame of image; and the non-occluded portion of the atleast partially occluded object is unrecognized in an N-th frame ofimage, N≥2; the method further comprises determining whether the atleast partially occluded object is outside field of view or becomes afully occluded object in the N-th frame of image; generating predictedsecond coordinates of the fully occluded object in the display apparatuscorresponding to the N-th frame of image upon a determination that theat least partially occluded object is still inside field of view; anddisplaying the simulated image at a position having the predicted secondcoordinates in the display apparatus.

Optionally, the non-occluded portion of the at least partially occludedobject is recognized in at least one frame of image from a first frameof image to an (N−1)-th frame of image; and the non-occluded portion ofthe at least partially occluded object is unrecognized in an N-th frameof image, N ≥2; the method further comprises generating predicted secondcoordinates of the at least partially occluded object in the displayapparatus corresponding to the N-th frame of image; determining whetherthe at least partially occluded object is outside field of view based onthe predicted second coordinates; displaying the simulated image at aposition having the predicted second coordinates in the displayapparatus upon a determination that the at least partially occludedobject is still inside field of view but becomes unrecognizable or fullyoccluded.

Optionally, the non-occluded portion of the at least partially occludedobject is recognized from an i-th frame of image to a j-th frame ofimage, i<j<N; and the non-occluded portion of the at least partiallyoccluded object is unrecognized from a (j+1)-th frame of image to anN-th frame of image, N≥4; the method further comprises generatingpredicted second coordinates of the at least partially occluded objectin the display apparatus corresponding to the N-th frame of imageaccording to Equation (5) and Equation (6):

$\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$

wherein (u_(n), v_(n)) is the second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image; (u_(i), v_(i)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the j-th frame of image; m₁=j−i+1; and m₂=n−j.

Optionally, the method further comprises displaying the simulated imageat a position having the second coordinates in the display apparatus.

Optionally, the method further comprises obtaining the input image usinga camera.

In one aspect, the present invention provides an apparatus for trackingan at least partially occluded object, comprising a memory; and one ormore processors; wherein the memory and the one or more processors areconnected with each other; and the memory stores computer-executableinstructions for controlling the one or more processors to recognize anon-occluded portion of the at least partially occluded object in aninput image; generate a simulated image of the at least partiallyoccluded object based on features of the non-occluded portion extractedfrom the input image; determine first coordinates of the at leastpartially occluded object in a first coordinate system; and convert thefirst coordinates of the at least partially occluded object in the firstcoordinate system into second coordinates in a second coordinate systemdefined in a display apparatus.

Optionally, the memory further stores computer-executable instructionsfor controlling the one or more processors to classify the non-occludedportion using classifiers trained using extracted features of portionsof a plurality of training objects; wherein the simulated image is animage of one of the plurality of training objects, a portion of whichcomprising extracted features closest matching the features of thenon-occluded portion extracted from the input image.

Optionally, the memory further stores computer-executable instructionsfor controlling the one or more processors to convert the firstcoordinates of the at least partially occluded object in the firstcoordinate system into third coordinates in a third coordinate system;and convert the third coordinates in the third coordinate system intothe second coordinates in the second coordinate system defined in thedisplay apparatus; wherein the first coordinate system is athree-dimensional coordinate system; and the third coordinate system isa two-dimensional coordinate system on a plane substantially parallel toan imaging plane of the display apparatus.

Optionally, the memory further stores computer-executable instructionsfor controlling the one or more processors to convert the firstcoordinates of the at least partially occluded object in the firstcoordinate system into the third coordinates in the third coordinatesystem according to Equation (1) and Equation (2) when the imaging planeof the display apparatus is substantially parallel to a coordinate planeof the first coordinate system:

$\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (x1, y1) stands forcoordinate differences between the first coordinates and a referencecoordinate of a reference point in the first coordinate system; f standsfor a length of an orthographic projection of a distance between theimaging plane and the reference point on a coordinate axis in the firstcoordinate system perpendicular to the imaging plane; and l stands for alength of an orthographic projection of a distance between the at leastpartially occluded object and the reference point on the coordinate axisin the first coordinate system perpendicular to the imaging plane.

Optionally, the reference point is a position of a view zone of a viewerin a vehicle.

Optionally, the memory further stores computer-executable instructionsfor controlling the one or more processors to convert the thirdcoordinates in the third coordinate system into the second coordinatesin the second coordinate system defined in the display apparatusaccording to Equation (3) and Equation (4):

$\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (u, v) stands for thesecond coordinates; (u₀, v₀) stands for a reference coordinate of areference point in the third coordinate system; dx stands for a realdistance representing a first pixel pitch along a first direction of thedisplay apparatus; dy stands for a real distance representing a secondpixel pitch along a second direction of the display apparatus.

Optionally, the reference point is a position of an orthographicprojection of a view zone of a viewer in a vehicle on the imaging plane.

Optionally, the non-occluded portion of the at least partially occludedobject is recognized in at least one frame from a first frame of imageto an (N−1)-th frame of image; and the non-occluded portion of the atleast partially occluded object is unrecognized in an N-th frame ofimage, N≥2; wherein the memory further stores computer-executableinstructions for controlling the one or more processors to determinewhether the at least partially occluded object is outside field of viewor becomes a fully occluded object in the N-th frame of image; andgenerate predicted second coordinates of the fully occluded object inthe display apparatus corresponding to the N-th frame of image upon adetermination that the at least partially occluded object is stillinside field of view; wherein the apparatus further comprises a displayapparatus configured to display the simulated image at a position havingthe predicted second coordinates in the display apparatus.

Optionally, the non-occluded portion of the at least partially occludedobject is recognized in at least one frame from a first frame of imageto an (N−1)-th frame of image; and the non-occluded portion of the atleast partially occluded object is unrecognized in an N-th frame ofimage, N≥2; wherein the memory further stores computer-executableinstructions for controlling the one or more processors to generatepredicted second coordinates of the at least partially occluded objectin the display apparatus corresponding to the N-th frame of image; anddetermine whether the at least partially occluded object is outsidefield of view based on the predicted second coordinates; wherein theapparatus further comprises a display apparatus configured to displaythe simulated image at a position having the predicted secondcoordinates in the display apparatus upon a determination that the atleast partially occluded object is still inside field of view butbecomes unrecognizable or fully occluded.

Optionally, the non-occluded portion of the at least partially occludedobject is recognized from an i-th frame of image to a j-th frame ofimage, i<j<N; and the non-occluded portion of the at least partiallyoccluded object is unrecognized from a (j+1)-th frame of image to anN-th frame of image, N≥4; wherein the memory further storescomputer-executable instructions for controlling the one or moreprocessors to generate predicted second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image according to Equation (5) and Equation (6):

$\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$

wherein (u_(n), v_(n)) is the second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image; (u_(i), v_(i)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the j-th frame of image; m₁=j−i+1; and m₂=n−j.

Optionally, the apparatus further comprises a camera configured toobtain the input image.

Optionally, the apparatus further comprises a display apparatusconfigured to display the simulated image at a position having thesecond coordinates in the display apparatus.

In one aspect, the present invention provides a vehicle comprising theapparatus for tracking an at least partially occluded object describedherein.

Optionally, the apparatus further comprises a camera configured toobtain the input image; wherein a distance between the camera and groundis equal to or less than a threshold distance.

Optionally, the memory further stores computer-executable instructionsfor controlling the one or more processors to generate an alarm signalbased on a critical condition being met.

Optionally, the memory further stores computer-executable instructionsfor controlling the one or more processors to determine whether thecritical condition is met based on information comprising a drivingdirection and the distance between the camera and the at least partiallyoccluded object.

In one aspect, the present invention provides a computer-program productcomprising a non-transitory tangible computer-readable medium havingcomputer-readable instructions thereon, the computer-readableinstructions being executable by a processor to cause the processor toperform recognizing a non-occluded portion of an at least partiallyoccluded object in an input image; generating a simulated image of theat least partially occluded object based on features of the non-occludedportion extracted from the input image; determining first coordinates ofthe at least partially occluded object in a first coordinate system; andconverting the first coordinates of the at least partially occludedobject in the first coordinate system into second coordinates in asecond coordinate system defined in a display apparatus.

BRIEF DESCRIPTION OF THE FIGURES

The following drawings are merely examples for illustrative purposesaccording to various disclosed embodiments and are not intended to limitthe scope of the present invention.

FIG. 1 is a schematic diagram illustrating an apparatus for tracking anat least partially occluded object in some embodiments according to thepresent disclosure.

FIG. 2 is a schematic diagram illustrating a vehicle having an apparatusfor tracking an at least partially occluded object in some embodimentsaccording to the present disclosure.

FIG. 3 is a schematic diagram illustrating a vehicular anti-collisionapparatus in some embodiments according to the present disclosure.

FIG. 4 is a flow chart illustrating a method for tracking an at leastpartially occluded object in some embodiments according to the presentdisclosure.

FIG. 5 illustrates a process of generating a simulated image of an atleast partially occluded object in some embodiments according to thepresent disclosure.

FIG. 6 illustrates coordinate systems for tracking an at least partiallyoccluded object in some embodiments according to the present disclosure.

FIG. 7 illustrates an imaging plane of a display apparatus and an atleast partially occluded object in a coordinate system in someembodiments according to the present disclosure.

FIG. 8 is a flow chart illustrating a method for tracking an at leastpartially occluded object in some embodiments according to the presentdisclosure.

FIG. 9 is a flow chart illustrating a method for tracking an at leastpartially occluded object in some embodiments according to the presentdisclosure.

DETAILED DESCRIPTION

The disclosure will now be described more specifically with reference tothe following embodiments. It is to be noted that the followingdescriptions of some embodiments are presented herein for purpose ofillustration and description only. It is not intended to be exhaustiveor to be limited to the precise form disclosed.

When a driver is driving a vehicle on a road, e.g., a multi-lane road,vehicles in adjacent lanes often obscure the driver's view. When thisoccurs, the object (such as a pedestrian, another vehicle, a bike, etc.)behind the vehicles in adjacent lanes is partially occluded or sometimescompletely occluded by the vehicles in adjacent lanes. The driver oftenis not aware of the occluded object. When the occluded objectsubsequently moves toward the driving direction of the vehicle, anaccident is prone to happen, particularly when the object is relativelyclose to the vehicle or moving toward a path along the drivingdirection.

Accordingly, the present disclosure provides, inter alia, a method fortracking an at least partially occluded object, an apparatus fortracking an at least partially occluded object, a vehicle, and acomputer-program product that substantially obviate one or more of theproblems due to limitations and disadvantages of the related art. In oneaspect, the present disclosure provides an apparatus for tracking an atleast partially occluded object. In some embodiments, the method oftracking an at least partially occluded object includes recognizing anon-occluded portion of the at least partially occluded object in aninput image; generating a simulated image of the at least partiallyoccluded object based on features of the non-occluded portion extractedfrom the input image; determining first coordinates of the at leastpartially occluded object in a first coordinate system; and convertingthe first coordinates of the at least partially occluded object in thefirst coordinate system into second coordinates in a second coordinatesystem defined in a display apparatus. Optionally, the first coordinatesystem is a coordinate system defined in a three-dimensional space.Optionally, the first coordinate system is a three-dimensionalcoordinate system. Optionally, the second coordinate system defined inthe display apparatus is a two-dimensional coordinate system.Optionally, the second coordinate system defined in the displayapparatus is a three-dimensional coordinate system.

As used herein, the term “occluded” refers to an object being partiallyor completely concealed by another object when viewed in a certain filedof view. As used herein, the term “non-occluded portion” of an at leastpartially occluded object refers to a portion of an object that is notconcealed by another object when viewed in the certain filed of view.

FIG. 1 is a schematic diagram illustrating an apparatus for tracking anat least partially occluded object in some embodiments according to thepresent disclosure. Referring to FIG. 1, the apparatus for tracking anat least partially occluded object 10 in some embodiments includes animage obtaining module 100 configured to obtain an input image, an imagerecognition module 200 configured to recognize a non-occluded portion ofthe at least partially occluded object in an input image, and generate asimulated image of the at least partially occluded object based onfeatures of the non-occluded portion extracted from the input image, anda coordinate determination module 300 configured to determine firstcoordinates of the at least partially occluded object in a firstcoordinate system, and convert the first coordinates of the at leastpartially occluded object in the first coordinate system into secondcoordinates in a second coordinate system defined in a displayapparatus. Optionally, the coordinate determination module 300 isconfigured to convert the first coordinates of the at least partiallyoccluded object in the first coordinate system into third coordinates ina third coordinate system, and convert the third coordinates in thethird coordinate system into the second coordinates in the secondcoordinate system defined in the display apparatus. Optionally, thefirst coordinate system is a three-dimensional coordinate system such asa world coordinate system. Optionally, the third coordinate system is atwo-dimensional coordinate system on a plane substantially parallel toan imaging plane of the display apparatus. Optionally, the imaging planeis an imaging plane of a virtual image, thus the imaging plane is avirtual imaging plane. Optionally, the imaging plane is an imaging planeof a real image.

In some embodiments, the coordinate determination module 300 isconfigured to classify the non-occluded portion using classifierstrained using extracted features of portions of a plurality of trainingobjects. Optionally, the simulated image is an image of one of theplurality of training objects, a portion of which having extractedfeatures closest matching the features of the non-occluded portionextracted from the input image. As used herein, the term “closestmatching” refers to having a best or one of the best similaritymeasurement for one or more particular similarity measures.

In some embodiments, the coordinate determination module 300 isconfigured to convert the first coordinates of the at least partiallyoccluded object in the first coordinate system into the thirdcoordinates in the third coordinate system according to Equation (1) andEquation (2) when the imaging plane of the display apparatus issubstantially parallel to a coordinate plane of the first coordinatesystem

$\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (x1, y1) stands forcoordinate differences between the first coordinates and a referencecoordinate of a reference point in the first coordinate system; f standsfor a length of an orthographic projection of a distance between theimaging plane and the reference point on a coordinate axis in the firstcoordinate system perpendicular to the imaging plane; and l stands for alength of an orthographic projection of a distance between the at leastpartially occluded object and the reference point on the coordinate axisin the first coordinate system perpendicular to the imaging plane.Optionally, the reference point is a position of a view zone (e.g.,human eyes) of a viewer (e.g., the driver) in a vehicle. Optionally, fcan be expressed as a distance between an orthographic projection of theimaging surface on a coordinate axis in the first coordinate systemperpendicular to the imaging surface and an orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface; and l can be expressed as adistance between an orthographic projection of the at least partiallyoccluded object on the coordinate axis in the first coordinate systemperpendicular to the imaging surface and the orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface.

In some embodiments, the coordinate determination module 300 isconfigured to convert the third coordinates in the third coordinatesystem into the second coordinates in the second coordinate systemdefined in the display apparatus according to Equation (3) and Equation(4):

$\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (u, v) stands for thesecond coordinates; (u₀, v₀) stands for a reference coordinate of areference point in the third coordinate system; dx stands for a realdistance representing a first pixel pitch along a first direction (e.g.,a row direction) of the display apparatus; dy stands for a real distancerepresenting a second pixel pitch along a second direction (e.g., acolumn direction) of the display apparatus. Optionally, the referencepoint is a position of an orthographic projection of a view zone of aviewer in a vehicle on the imaging plane. Optionally, dx can beexpressed as a real distance representing a size of a pixel along thefirst direction (e.g., the row direction) of the display apparatus; anddy stands for a real distance representing a size of a pixel along thesecond direction (e.g., the column direction) of the display apparatus.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized in at least one frame from a first frameof image to an (N−1)-th frame of image; and the non-occluded portion ofthe at least partially occluded object is unrecognized in an N-th frameof image, N≥2. Optionally, the coordinate determination module 300 isconfigured to determine whether the at least partially occluded objectis outside field of view or becomes a fully occluded object in the N-thframe of image; and generate predicted second coordinates of the fullyoccluded object in the display apparatus corresponding to the N-th frameof image upon a determination that the at least partially occludedobject is still inside field of view. Optionally, the apparatus furtherincludes a display apparatus configured to display the simulated imageat a position having the predicted second coordinates in the displayapparatus. As used herein, the term “field of view” refers to an extentof three-dimensional space covered and “viewed” by a view zone, e.g., animage obtaining module such as a camera, a human eye, etc.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized in at least one frame from a first frameof image to an (N−1)-th frame of image; and the non-occluded portion ofthe at least partially occluded object is unrecognized in an N-th frameof image, N≥2. Optionally, the coordinate determination module 300 isconfigured to generate predicted second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image; and determine whether the at least partiallyoccluded object is outside field of view based on the predicted secondcoordinates. Optionally, the apparatus further includes a displayapparatus configured to display the simulated image at a position havingthe predicted second coordinates in the display apparatus upon adetermination that the at least partially occluded object is stillinside field of view but becomes unrecognizable or fully occluded.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized from an i-th frame of image to a j-thframe of image, i<j<N; and the non-occluded portion of the at leastpartially occluded object is unrecognized from a (j+1)-th frame of imageto an N-th frame of image. Optionally, N≥4. Optionally, the coordinatedetermination module 300 is configured to generate predicted secondcoordinates of the at least partially occluded object in the displayapparatus corresponding to the N-th frame of image according to Equation(5) and Equation (6):

$\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$

wherein (u_(n), v_(n)) is the second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image; (u_(i), v_(i)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the j-th frame of image; m₁=j−i+1; and m₂=n−j. Optionally, theapparatus further includes a camera (e.g., a video camera) configured toobtain the input image. Optionally, the apparatus further includes adisplay apparatus configured to display the simulated image at aposition having the second coordinates in the display apparatus.Optionally, the camera is a monocular camera. Optionally, the camera isa binocular camera.

FIG. 2 is a schematic diagram illustrating a vehicle having an apparatusfor tracking an at least partially occluded object in some embodimentsaccording to the present disclosure. In some embodiments, the apparatusfor tracking an at least partially occluded object in some embodimentsis installed in a vehicle. Optionally, the viewer is a driver of thevehicle. Optionally, the view zone is one or both eyes of the driver.Referring to FIG. 2, the apparatus for tracking an at least partiallyoccluded object 10 in some embodiments includes an image obtainingmodule 100, an image recognition module 200, a coordinate determinationmodule 300, and a display apparatus 400. The image obtaining module 100can be installed at any appropriate position of the vehicle. Forexample, the image obtaining module 100 (e.g., one or more cameras) canbe installed on one or both sides of the vehicle. In another example,the image obtaining module 100 (e.g., one or more cameras) can beinstalled on a chassis of the vehicle. Optionally, the image obtainingmodule 100 can be installed at one or both of a position 1 and aposition 2 of the vehicle. For example, the position 1 is at right frontof the vehicle, and the position 2 is at left front of the vehicle. Theat least partially occluded object may be an object at least partiallyoccluded by an obstacle (e.g., a nearby vehicle). Accordingly, the imageobtaining module 100 is disposed at a position so that the imageobtaining module 100 can capture a non-occluded portion of the at leastpartially occluded object even though the at least partially occludedobject is completely or partially absent in the driver's field of view.For example, the image obtaining module 100 has a field of viewdifferent from the driver's field of view. Optionally, the imageobtaining module 100 is disposed at a position relatively close toground as compared to the driver's eyes. In one example, the imageobtaining module 100 is disposed at a front side of the vehicle. Inanother example the image obtaining module 100 is disposed on thechassis close to the front of the vehicle. Optionally, a distancebetween the image obtaining module 100 and ground is equal to or lessthan a threshold distance. Optionally, the distance between the imageobtaining module 100 and ground less than a distance between theviewer's view zone (e.g., the driver's eyes) and ground.

In some embodiments, the coordinate determination module 300 isconfigured to determine whether the critical condition is met.Optionally, the determination is based on information including adriving direction of the vehicle and a distance between the camera andthe at least partially occluded object. Optionally, the coordinatedetermination module 300 is configured to generate an alarm signal basedon the critical condition being met. Optionally, the vehicle includes analarm device, and upon receiving the alarm signal, the alarm devicegenerates an alarm to a driver. Optionally, the alarm includes one or acombination of a textual alarm, a visual alarm, and an audio alarm.

FIG. 3 is a schematic diagram illustrating a vehicular anti-collisionapparatus in some embodiments according to the present disclosure.Referring to FIG. 3, the vehicular anti-collision apparatus 20 in someembodiments includes an image obtaining module 100, an image recognitionmodule 200, a coordinate determination module 300, and a displayapparatus 400.

In some embodiments, the apparatus for tracking an at least partiallyoccluded object includes a memory; and one or more processors. Thememory and the one or more processors are connected with each other; andthe memory stores computer-executable instructions for controlling theone or more processors to recognize a non-occluded portion of the atleast partially occluded object in an input image; generate a simulatedimage of the at least partially occluded object based on features of thenon-occluded portion extracted from the input image; determine firstcoordinates of the at least partially occluded object in a firstcoordinate system; and convert the first coordinates of the at leastpartially occluded object in the first coordinate system into secondcoordinates in a second coordinate system defined in a displayapparatus.

In some embodiments, the memory further stores computer-executableinstructions for controlling the one or more processors to classify thenon-occluded portion using classifiers trained using extracted featuresof portions of a plurality of training objects. Optionally, thesimulated image is an image of one of the plurality of training objects,a portion of which comprising extracted features closest matching thefeatures of the non-occluded portion extracted from the input image.

In some embodiments, the memory further stores computer-executableinstructions for controlling the one or more processors to convert thefirst coordinates of the at least partially occluded object in the firstcoordinate system into third coordinates in a third coordinate system;and convert the third coordinates in the third coordinate system intothe second coordinates in the second coordinate system defined in thedisplay apparatus. Optionally, the first coordinate system is athree-dimensional coordinate system, e.g., a world coordinate system.Optionally, the third coordinate system is a two-dimensional coordinatesystem on a plane substantially parallel to an imaging plane of thedisplay apparatus. Optionally, the imaging plane is an imaging plane ofa virtual image, thus the imaging plane is a virtual imaging plane.Optionally, the imaging plane is an imaging plane of a real image.

In some embodiments, the memory further stores computer-executableinstructions for controlling the one or more processors to convert thefirst coordinates of the at least partially occluded object in the firstcoordinate system into the third coordinates in the third coordinatesystem according to Equation (1) and Equation (2) when the imaging planeof the display apparatus is substantially parallel to a coordinate planeof the first coordinate system:

$\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (x1, y1) stands forcoordinate differences between the first coordinates and a referencecoordinate of a reference point in the first coordinate system; f standsfor a length of an orthographic projection of a distance between theimaging plane and the reference point on a coordinate axis in the firstcoordinate system perpendicular to the imaging plane; and l stands for alength of an orthographic projection of a distance between the at leastpartially occluded object and the reference point on the coordinate axisin the first coordinate system perpendicular to the imaging plane.Optionally, the reference point is a position of a view zone (e.g.,human eyes) of a viewer (e.g., the driver) in a vehicle. Optionally, fcan be expressed as a distance between an orthographic projection of theimaging surface on a coordinate axis in the first coordinate systemperpendicular to the imaging surface and an orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface; and l can be expressed as adistance between an orthographic projection of the at least partiallyoccluded object on the coordinate axis in the first coordinate systemperpendicular to the imaging surface and the orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface.

In some embodiments, the memory further stores computer-executableinstructions for controlling the one or more processors to convert thethird coordinates in the third coordinate system into the secondcoordinates in the second coordinate system defined in the displayapparatus according to Equation (3) and Equation (4):

$\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (u, v) stands for thesecond coordinates; (u₀, v₀) stands for a reference coordinate of areference point in the third coordinate system; dx stands for a realdistance representing a first pixel pitch along a first direction (e.g.,a row direction) of the display apparatus; dy stands for a real distancerepresenting a second pixel pitch along a second direction (e.g., acolumn direction) of the display apparatus. Optionally, the referencepoint is a position of an orthographic projection of a view zone of aviewer in a vehicle on the imaging plane. Optionally, dx can beexpressed as a real distance representing a size of a pixel along thefirst direction (e.g., the row direction) of the display apparatus; anddy stands for a real distance representing a size of a pixel along thesecond direction (e.g., the column direction) of the display apparatus.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized in at least one frame from a first frameof image to an (N−1)-th frame of image; and the non-occluded portion ofthe at least partially occluded object is unrecognized in an N-th frameof image, N≥2. Optionally, the memory further stores computer-executableinstructions for controlling the one or more processors to determinewhether the at least partially occluded object is outside field of viewor becomes a fully occluded object in the N-th frame of image; andgenerate predicted second coordinates of the fully occluded object inthe display apparatus corresponding to the N-th frame of image upon adetermination that the at least partially occluded object is stillinside field of view. Optionally, the apparatus further includes adisplay apparatus configured to display the simulated image at aposition having the predicted second coordinates in the displayapparatus.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized in at least one frame from a first frameof image to an (N−1)-th frame of image; and the non-occluded portion ofthe at least partially occluded object is unrecognized in an N-th frameof image, N≥2. Optionally, the memory further stores computer-executableinstructions for controlling the one or more processors to generatepredicted second coordinates of the at least partially occluded objectin the display apparatus corresponding to the N-th frame of image; anddetermine whether the at least partially occluded object is outsidefield of view based on the predicted second coordinates. Optionally, theapparatus further includes a display apparatus configured to display thesimulated image at a position having the predicted second coordinates inthe display apparatus upon a determination that the at least partiallyoccluded object is still inside field of view but becomes unrecognizableor fully occluded.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized from an i-th frame of image to a j-thframe of image, i<j<N; and the non-occluded portion of the at leastpartially occluded object is unrecognized from a (j+1)-th frame of imageto an N-th frame of image. Optionally, N≥4. Optionally, the memoryfurther stores computer-executable instructions for controlling the oneor more processors to generate predicted second coordinates of the atleast partially occluded object in the display apparatus correspondingto the N-th frame of image according to Equation (5) and Equation (6):

$\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$

wherein (u_(n), v_(n)) is the second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image; (u_(i), v_(i)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the j-th frame of image; m₁=j−i+1; and m₂=n−j. Optionally, theapparatus further includes a camera configured to obtain the inputimage. Optionally, the apparatus further includes a display apparatusconfigured to display the simulated image at a position having thesecond coordinates in the display apparatus.

In some embodiments, the memory further stores computer-executableinstructions for controlling the one or more processors to determinewhether the critical condition is met. Optionally, the determination isbased on information including a driving direction of the vehicle and adistance between the camera and the at least partially occluded object.Optionally, the memory further stores computer-executable instructionsfor controlling the one or more processors to generate an alarm signalbased on the critical condition being met. Optionally, a vehicle havingthe apparatus for tracking an at least partially occluded object furtherincludes an alarm device, and upon receiving the alarm signal, the alarmdevice generates an alarm to a driver. Optionally, the alarm includesone or a combination of a textual alarm, a visual alarm, and an audioalarm.

In another aspect, the present disclosure provides a method for trackingan at least partially occluded object. FIG. 4 is a flow chartillustrating a method for tracking an at least partially occluded objectin some embodiments according to the present disclosure. Referring toFIG. 4, the method in some embodiments includes recognizing anon-occluded portion of the at least partially occluded object in aninput image; generating a simulated image of the at least partiallyoccluded object based on features of the non-occluded portion extractedfrom the input image; determining first coordinates of the at leastpartially occluded object in a first coordinate system; and convertingthe first coordinates of the at least partially occluded object in thefirst coordinate system into second coordinates in a second coordinatesystem defined in a display apparatus. Optionally, the first coordinatesystem is a coordinate system defined in a three-dimensional space.Optionally, the first coordinate system is a three-dimensionalcoordinate system. Optionally, the second coordinate system defined inthe display apparatus is a two-dimensional coordinate system.Optionally, the second coordinate system defined in the displayapparatus is a three-dimensional coordinate system.

In some embodiments, the method further includes obtaining the inputimage using an image obtaining module (e.g., a camera). In someembodiments, the method further includes determining a distance betweenthe image obtaining module (e.g., a camera) and the at least partiallyoccluded object. Optionally, the method further includes determiningwhether a critical condition is met. Optionally, the determination isbased on information including a driving direction of the vehicle andthe distance between the image obtaining module and the at leastpartially occluded object. Optionally, the method further includesgenerating an alarm signal based on the critical condition being met.Optionally, the distance between the image obtaining module and the atleast partially occluded object is determined using an image obtained bya binocular camera.

The image obtaining module can be installed in any appropriate positionof the vehicle. For example, the image obtaining module (e.g., one ormore cameras) can be installed on one or both sides of the vehicle. Inanother example, the image obtaining module (e.g., one or more cameras)can be installed on a chassis of the vehicle. Optionally, the imageobtaining module can be installed at one or both of a right frontposition of the vehicle and a left front position of the vehicle. The atleast partially occluded object may be an object at least partiallyoccluded by an obstacle (e.g., a nearby vehicle). Accordingly, the imageobtaining module is disposed at a position so that the image obtainingmodule can capture a non-occluded portion of the at least partiallyoccluded object even though the at least partially occluded object iscompletely or partially absent in the driver's field of view. Forexample, the image obtaining module has a field of view different fromthe driver's field of view. Optionally, the image obtaining module isdisposed at a position relatively close to ground as compared to thedriver's eyes. In one example, the image obtaining module is disposed ata front side of the vehicle. In another example the image obtainingmodule is disposed on the chassis close to the front of the vehicle.Optionally, a distance between the image obtaining module and ground isequal to or less than a threshold distance. Optionally, the distancebetween the image obtaining module and ground less than a distancebetween the viewer's view zone (e.g., the driver's eyes) and ground.

The at least partially occluded object may be any appropriate objectsuch as a human, an animal, or any other types of objects (e.g., a wheelof a vehicle).

In some embodiments, the non-occluded portion of the at least partiallyoccluded object in the input image is recognized using a deep learningmethod. In some embodiments, the step of generating the simulated imageof the at least partially occluded object includes classifying thenon-occluded portion using classifiers trained using extracted featuresof portions of a plurality of training objects. Optionally, thesimulated image is an image of one of the plurality of training objects,a portion of which including extracted features closest matching thefeatures of the non-occluded portion extracted from the input image.Optionally, the simulated image

In some embodiments, the features of the non-occluded portion extractedfrom the input image includes a contour of the non-occluded portion ofthe at least partially occluded object, and the contour of thenon-occluded portion of the at least partially occluded object isclassified using contours of portions of a plurality of trainingobjects. A contour of a portion of the plurality of training objects isfound to have the closest match with the contour of the non-occludedportion of the at least partially occluded object. The image of the oneof the training objects having the portion whose contour has the closestmatch is chosen as the simulated image. In one example, a contour imageof the one of the training objects having the portion whose contour hasthe closest match is chosen as the simulated image. FIG. 5 illustrates aprocess of generating a simulated image of an at least partiallyoccluded object in some embodiments according to the present disclosure.Referring to FIG. 5, a driver D is driving a first vehicle V1, and apedestrian is partially occluded by a nearby vehicle V2. The lower bodyof the pedestrian P is not occluded in the field of view of an imageobtaining module 100. An input image is obtained by the image obtainingmodule 100. A non-occluded portion (e.g., the lower body) of the atleast partially occluded object (e.g., the pedestrian P) is recognizedin the input image. The contour of the non-occluded portion, e.g., thecontour of the lower part, is classified using contours of portions of aplurality of training objects, and is classified as a contour of a lowerpart of a human. Based on this classification, a simulated image S of ahuman (e.g., a contour image of a human) is generated and displayed in adisplay apparatus in vehicle V1. By recognizing the non-occluded portionof the at least partially occluded object and generating the simulatedimage of the at least partially occluded object, the driver D can beaware of the at least partially occluded object (e.g., the pedestrian Pbehind the nearby vehicle V2), even though the at least partiallyoccluded object is completely outside the driver D's field of view.Because the driver becomes aware of the presence of the at leastpartially occluded object (e.g., the pedestrian P), and the movement ofthe at least partially occluded object can be tracked, potentialaccidents such as a collision with the pedestrian can be avoided whenthe pedestrian is moving toward the vehicle V1. The present methodgreatly improves driving safety.

Any appropriate display apparatus may be used in the present method.Optionally, the display apparatus is a head-up display. Optionally, thedisplay apparatus includes a projection device. Optionally, the displayapparatus is an augmented display apparatus. Optionally, the displayapparatus is a virtual reality display apparatus.

In some embodiments, the head-up display includes a display unit and animaging optical system. Optionally, the display unit is configured todisplay vehicle status information such as vehicle speed and fuel level,as well as navigation information and alarm indication information.Optionally, the imaging optical system is configured to reflect lightemitted by the display unit onto the windshield of the vehicle, and thewindshield is configured to subsequently reflected to light to a viewzone of a viewer (e.g., the driver's eyes), thereby generating a virtualimage on an imaging plane. The driver can observe the virtual imagethrough the windshield.

Optionally, the display unit includes a back light module and a displaypanel. Optionally, the display panel is configured to display imageinformation transmitted from the electronic control unit of the vehicle,e.g., the vehicle status information such as vehicle speed and fuellevel, as well as navigation information and alarm indicationinformation. The back light module is configured to provide back lightto the display panel.

Various appropriate imaging optical system may be used in the presentdisplay apparatus. In one example, the imaging optical system includesone or a combination of a reflective mirror, a transflective mirror, anda prism. Optionally, the imaging optical system includes a reflectivemirror, this type of imaging optical system saves space.

In some embodiments, the second coordinates refer to coordinates of adesignated point of the simulated image of the at least partiallyoccluded object in the second coordinate system defined in a displayapparatus. The designated point may be any appropriate point. In oneexample, the designated point is a point in the simulated imagecorresponding to a central point of the at least partially occludedobject. In another example, the designated point is a point in thesimulated image corresponding to a mass center of the at least partiallyoccluded object. For example, a second coordinate (1, 1) in the secondcoordinate system indicates the designated point is at a pixel in afirst row and a first column of the display apparatus. In anotherexample, a second coordinate (3, 4) in the second coordinate systemindicates the designated point is at a pixel in a third row and a fourthcolumn of the display apparatus.

In some embodiments, the first coordinate system is a three-dimensionalcoordinate system, e.g., a world coordinate system. Optionally, thefirst coordinate system is a pre-set coordinate system. Optionally, thefirst coordinate system is an image obtaining module coordinate system,e.g., a camera coordinate system. FIG. 6 illustrates coordinate systemsfor tracking an at least partially occluded object in some embodimentsaccording to the present disclosure. Referring to FIG. 6, the firstcoordinate system in some embodiments is an image obtaining modulecoordinate system depicted using X1, Y1, and Z1 axes, e.g., athree-dimensional coordinate system. As shown in FIG. 6, the origin O1of the image obtaining module coordinate system is the position of theimage obtaining module 100. Optionally, the X1-O1-Y1 plane isperpendicular to the horizontal plane. The Z1 axis of the imageobtaining module coordinate system is parallel to the horizontal plane.

Optionally, when the first coordinate system is the image obtainingmodule coordinate system, e.g., the camera coordinate system, the firstcoordinates of the at least partially occluded object in a firstcoordinate system can be determined directly. Optionally, when the firstcoordinate system is the image obtaining module coordinate system, e.g.,the camera coordinate system, a distance between the image obtainingmodule and the at least partially occluded object can be firstdetermined, and the first coordinates of the at least partially occludedobject in a first coordinate system can be determined based on thedistance between the image obtaining module and the at least partiallyoccluded object.

Alternatively, the first coordinate system is a driver coordinate systemdepicted using X2, Y2, and Z2 axes. Referring to FIG. 6, the firstcoordinate system in some embodiments is a driver coordinate system. Theorigin O2 of the driver coordinate system is the position of the driverD. Optionally, the X2-O2-Y2 plane is perpendicular to the horizontalplane. The Z2 axis of the driver coordinate system is parallel to thehorizontal plane.

Optionally, when the first coordinate system is a coordinate systemother than the image obtaining module coordinate system, coordinates ofthe at least partially occluded object in the image obtaining modulecoordinate system can be first determined, followed by converting thecoordinates of the at least partially occluded object in the imageobtaining module coordinate system into the first coordinates in thefirst coordinate system.

In some embodiments, the step of converting the first coordinates of theat least partially occluded object in the first coordinate system intothe second coordinates in the second coordinate system defined in thedisplay apparatus includes converting the first coordinates of the atleast partially occluded object in the first coordinate system intothird coordinates in a third coordinate system; and converting the thirdcoordinates in the third coordinate system into the second coordinatesin the second coordinate system defined in the display apparatus.Optionally, the first coordinate system is a three-dimensionalcoordinate system, and the third coordinate system is a two-dimensionalcoordinate system on a plane substantially parallel to an imaging planeof the display apparatus. Optionally, the imaging plane is an imagingplane of a virtual image, thus the imaging plane is a virtual imagingplane. Optionally, the imaging plane is an imaging plane of a realimage.

FIG. 7 illustrates an imaging plane of a display apparatus and an atleast partially occluded object in a coordinate system in someembodiments according to the present disclosure. Referring to FIG. 7,the third coordinate system 101 is a two-dimensional coordinate systemon a plane substantially parallel to an imaging plane of the displayapparatus. Typically, the third coordinate system 101 is on a planeperpendicular to the horizontal plane. Any appropriate point in thethird coordinate system 101 may be used as the origin of the thirdcoordinate system 101. In some embodiments, the first coordinates of theat least partially occluded object in the first coordinate system can befirst converted into third coordinates in a third coordinate system 101.Subsequently, the third coordinates in the third coordinate system 101can be converted into the second coordinates in the second coordinatesystem defined in the display apparatus. Optionally, the thirdcoordinate system 101 is on a plane parallel to a coordinate axis of thefirst coordinate system. For example, in FIG. 7, the third coordinatesystem 101 is on a plane parallel to the X2-O2-Y2 plane of the firstcoordinate system (which is a driver coordinate system). Optionally, thethird coordinate system 101 is on a plane non-parallel to any coordinateplane of the first coordinate system.

In some embodiments, the imaging plane of the display apparatus issubstantially parallel to a coordinate plane of the first coordinatesystem, e.g., the third coordinate system 101 is on a plane parallel tothe X2-O2-Y2 coordinate plane of the first coordinate system.Optionally, the step of converting the first coordinates of the at leastpartially occluded object in the first coordinate system into the thirdcoordinates in the third coordinate system is performed according toEquation (1) and Equation (2):

$\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (x1, y1) stands forcoordinate differences between the first coordinates and a referencecoordinate of a reference point in the first coordinate system; f standsfor a length of an orthographic projection of a distance between theimaging plane and the reference point on a coordinate axis in the firstcoordinate system perpendicular to the imaging plane; and l stands for alength of an orthographic projection of a distance between the at leastpartially occluded object and the reference point on the coordinate axisin the first coordinate system perpendicular to the imaging plane.Optionally, the reference point is a position of a view zone (e.g.,human eyes) of a viewer (e.g., the driver) in a vehicle. Optionally, fcan be expressed as a distance between an orthographic projection of theimaging surface on a coordinate axis in the first coordinate systemperpendicular to the imaging surface and an orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface; and l can be expressed as adistance between an orthographic projection of the at least partiallyoccluded object on the coordinate axis in the first coordinate systemperpendicular to the imaging surface and the orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface.

In some embodiments, the step of converting the third coordinates in thethird coordinate system into the second coordinates in the secondcoordinate system defined in the display apparatus is performedaccording to Equation (3) and Equation (4):

$\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (u, v) stands for thesecond coordinates; (u₀, v₀) stands for a reference coordinate of areference point in the third coordinate system; dx stands for a realdistance representing a first pixel pitch along a first direction (e.g.,a row direction) of the display apparatus; dy stands for a real distancerepresenting a second pixel pitch along a second direction (e.g., acolumn direction) of the display apparatus. Optionally, the referencepoint is a position of an orthographic projection of a view zone of aviewer in a vehicle on the imaging plane. Optionally, dx can beexpressed as a real distance representing a size of a pixel along thefirst direction (e.g., the row direction) of the display apparatus; anddy stands for a real distance representing a size of a pixel along thesecond direction (e.g., the column direction) of the display apparatus.

In some embodiments, the method further includes displaying thesimulated image at a position having the second coordinates in thedisplay apparatus. In some embodiments, the second coordinates displayedin the display apparatus refer to coordinates of a designated point ofthe simulated image of the at least partially occluded object in thesecond coordinate system defined in a display apparatus. The designatedpoint may be any appropriate point. In one example, the designated pointis a point in the simulated image corresponding to a central point ofthe at least partially occluded object. In another example, thedesignated point is a point in the simulated image corresponding to amass center of the at least partially occluded object. For example, asecond coordinate (1, 1) in the second coordinate system indicates thedesignated point is at a pixel in a first row and a first column of thedisplay apparatus. In another example, a second coordinate (3, 4) in thesecond coordinate system indicates the designated point is at a pixel ina third row and a fourth column of the display apparatus.

In some embodiments, the image obtaining module is configured to obtaina plurality of frames of images, and the input image is one of theplurality of frames of images. Optionally, in one or more of theplurality of frames of images, the at least partially occluded object isunrecognized. For example, the at least partially occluded object hasmoved outside field of view of the image obtaining module or has becomefully occluded. Optionally, the at least partially occluded object inthe one or more of the plurality of frames of images where it isunrecognized can be tracked based on information obtained in one or moreof the plurality of frames of images where the at least partiallyoccluded object is recognized. By tracking the at least partiallyoccluded object throughout all frames of images, the movement of the atleast partially occluded object can be monitored more accurately.

FIG. 8 is a flow chart illustrating a method for tracking an at leastpartially occluded object in some embodiments according to the presentdisclosure. Referring to FIG. 8, the method in some embodiments includesdetermining whether the non-occluded portion of the at least partiallyoccluded object is unrecognized in an N-th frame of image, N ≥2. Upon adetermination that the non-occluded portion of the at least partiallyoccluded object is unrecognized in the N-th frame of image, the methodthen determines whether the non-occluded portion of the at leastpartially occluded object is recognized in at least one frame from afirst frame of image to an (N−1)-th frame of image. Upon a determinationthat the non-occluded portion of the at least partially occluded objectis recognized in at least one frame from a first frame of image to an(N−1)-th frame of image, the method further includes determining whetherthe at least partially occluded object is outside field of view orbecomes a fully occluded object in the N-th frame of image. Upon adetermination that the at least partially occluded object is stillinside field of view, the method then includes generating predictedsecond coordinates of the fully occluded object in the display apparatuscorresponding to the N-th frame of image upon a determination that theat least partially occluded object is still inside field of view; anddisplaying the simulated image at a position having the predicted secondcoordinates in the display apparatus.

Optionally, whether the at least partially occluded object is outsidefield of view can be determined based on a determination of whethersecond coordinates of the at least partially occluded objectcorresponding to the (N−1)-th frame of image is still within the displayrange of the display apparatus. Optionally, based on a determinationthat the second coordinates of the at least partially occluded objectcorresponding to the (N−1)-th frame of image is still within the displayrange of the display apparatus, it is determined that the at leastpartially occluded object is inside field of view. Optionally, based ona determination that the second coordinates of the at least partiallyoccluded object corresponding to the (N−1)-th frame of image is notwithin the display range of the display apparatus, it is determined thatthe at least partially occluded object is outside field of view.

FIG. 9 is a flow chart illustrating a method for tracking an at leastpartially occluded object in some embodiments according to the presentdisclosure. Referring to FIG. 8, the method in some embodiments includesdetermining whether the non-occluded portion of the at least partiallyoccluded object is unrecognized in an N-th frame of image, N≥2. Upon adetermination that the non-occluded portion of the at least partiallyoccluded object is unrecognized in the N-th frame of image, the methodthen determines whether the non-occluded portion of the at leastpartially occluded object is recognized in at least one frame from afirst frame of image to an (N−1)-th frame of image. Upon a determinationthat the non-occluded portion of the at least partially occluded objectis recognized in at least one frame from a first frame of image to an(N−1)-th frame of image, the method further includes generatingpredicted second coordinates of the at least partially occluded objectin the display apparatus corresponding to the N-th frame of image; anddetermining whether the at least partially occluded object is outsidefield of view based on the predicted second coordinates. Upon adetermination that the at least partially occluded object is insidefield of view based on the predicted second coordinates, the method thenfurther includes displaying the simulated image at a position having thepredicted second coordinates in the display apparatus upon adetermination that the at least partially occluded object is stillinside field of view but becomes unrecognizable or fully occluded.

Optionally, whether the at least partially occluded object is outsidefield of view can be determined based on a determination of whethersecond coordinates of the at least partially occluded objectcorresponding to the N-th frame of image is still within the displayrange of the display apparatus. Optionally, based on a determinationthat the second coordinates of the at least partially occluded objectcorresponding to the N-th frame of image is still within the displayrange of the display apparatus, it is determined that the at leastpartially occluded object is inside field of view. Optionally, based ona determination that the second coordinates of the at least partiallyoccluded object corresponding to the N-th frame of image is not withinthe display range of the display apparatus, it is determined that the atleast partially occluded object is outside field of view.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized from an i-th frame of image to a j-thframe of image, i<j<N; and the non-occluded portion of the at leastpartially occluded object is unrecognized from a (j+1)-th frame of imageto an N-th frame of image. Optionally, N≥4. Optionally, the methodfurther includes generating predicted second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image according to Equation (5) and Equation (6):

$\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$

wherein (u_(n), v_(n)) is the second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image; (u_(i), v_(i)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the j-th frame of image; m₁=j−i+1; and m₂=n−j.

In another aspect, the present disclosure further provides a vehiclehaving an apparatus for tracking an at least partially occluded objectdescribed herein or a vehicular anti-collision apparatus describedherein. Optionally, a distance between the camera and ground is equal toor less than a threshold distance. Optionally, the apparatus fortracking an at least partially occluded object is configured to generatean alarm signal based on a critical condition being met, for example,the memory further stores computer-executable instructions forcontrolling the one or more processors to generate an alarm signal basedon a critical condition being met. Optionally, the apparatus fortracking an at least partially occluded object is configured todetermine whether the critical condition is met based on informationincluding a driving direction and a distance between the camera and theat least partially occluded object, e.g., the memory further storescomputer-executable instructions for controlling the one or moreprocessors to determine whether the critical condition is met based oninformation including a driving direction and a distance between thecamera and the at least partially occluded object.

Optionally, when the at least partially occluded object is in a path ofthe vehicle along the driving direction, and the distance between thecamera and the at least partially occluded object is less than athreshold distance, the apparatus for tracking an at least partiallyoccluded object is configured to generate the alarm signal. Optionally,when the at least partially occluded object is outside the path of thevehicle along the driving direction, the alarm signal is not generated.Optionally, the threshold distance is greater than a product of thedriving speed and a reaction time of the driver.

Optionally, the vehicle further includes an alarm device, and uponreceiving the alarm signal, the alarm device generates an alarm to adriver. Optionally, the alarm includes one or a combination of a textualalarm, a visual alarm, and an audio alarm.

By generating the alarm signal, the driver becomes aware of the presenceof the at least partially occluded object, and the movement of the atleast partially occluded object can be tracked, potential accidents suchas a collision with the pedestrian can be avoided when the pedestrian ismoving toward the vehicle. Driving safety can be greatly enhanced in thepresent vehicle.

In another aspect, the present disclosure provides a computer-programproduct comprising a non-transitory tangible computer-readable mediumhaving computer-readable instructions thereon. In some embodiments, thecomputer-readable instructions being executable by a processor to causethe processor to perform recognizing a non-occluded portion of an atleast partially occluded object in an input image; generating asimulated image of the at least partially occluded object based onfeatures of the non-occluded portion extracted from the input image;determining first coordinates of the at least partially occluded objectin a first coordinate system; and converting the first coordinates ofthe at least partially occluded object in the first coordinate systeminto second coordinates in a second coordinate system defined in adisplay apparatus. Optionally, the first coordinate system is acoordinate system defined in a three-dimensional space. Optionally, thefirst coordinate system is a three-dimensional coordinate system.Optionally, the second coordinate system defined in the displayapparatus is a two-dimensional coordinate system. Optionally, the secondcoordinate system defined in the display apparatus is athree-dimensional coordinate system.

In some embodiments, the computer-readable instructions being executableby a processor to cause the processor to further perform generating thesimulated image of the at least partially occluded object comprisesclassifying the non-occluded portion using classifiers trained usingextracted features of portions of a plurality of training objects.Optionally, the simulated image is an image of one of the plurality oftraining objects, a portion of which comprising extracted featuresclosest matching the features of the non-occluded portion extracted fromthe input image.

In some embodiments, the computer-readable instructions being executableby a processor to cause the processor to further perform converting thefirst coordinates of the at least partially occluded object in the firstcoordinate system into third coordinates in a third coordinate system;and converting the third coordinates in the third coordinate system intothe second coordinates in the second coordinate system defined in thedisplay apparatus. Optionally, the first coordinate system is athree-dimensional coordinate system. Optionally, the third coordinatesystem is a two-dimensional coordinate system on a plane substantiallyparallel to an imaging plane of the display apparatus. Optionally, theimaging plane is an imaging plane of a virtual image, thus the imagingplane is a virtual imaging plane. Optionally, the imaging plane is animaging plane of a real image.

In some embodiments, the computer-readable instructions being executableby a processor to cause the processor to perform converting the firstcoordinates of the at least partially occluded object in the firstcoordinate system into the third coordinates in the third coordinatesystem is performed according to Equation (1) and Equation (2) when theimaging plane of the display apparatus is substantially parallel to acoordinate plane of the first coordinate system:

$\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (x1, y1) stands forcoordinate differences between the first coordinates and a referencecoordinate of a reference point in the first coordinate system; f standsfor a length of an orthographic projection of a distance between theimaging plane and the reference point on a coordinate axis in the firstcoordinate system perpendicular to the imaging plane; and l stands for alength of an orthographic projection of a distance between the at leastpartially occluded object and the reference point on the coordinate axisin the first coordinate system perpendicular to the imaging plane.Optionally, the reference point is a position of a view zone (e.g.,human eyes) of a viewer (e.g., the driver) in a vehicle. Optionally, fcan be expressed as a distance between an orthographic projection of theimaging surface on a coordinate axis in the first coordinate systemperpendicular to the imaging surface and an orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface; and l can be expressed as adistance between an orthographic projection of the at least partiallyoccluded object on the coordinate axis in the first coordinate systemperpendicular to the imaging surface and the orthographic projection ofthe reference point on the coordinate axis in the first coordinatesystem perpendicular to the imaging surface.

In some embodiments, the computer-readable instructions being executableby a processor to cause the processor to perform converting the thirdcoordinates in the third coordinate system into the second coordinatesin the second coordinate system defined in the display apparatus isperformed according to Equation (3) and Equation (4):

$\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$

wherein (x2, y2) stands for the third coordinates; (u, v) stands for thesecond coordinates; (u₀, v₀) stands for a reference coordinate of areference point in the third coordinate system; dx stands for a realdistance representing a first pixel pitch along a first direction (e.g.,a row direction) of the display apparatus; dy stands for a real distancerepresenting a second pixel pitch along a second direction (e.g., acolumn direction) of the display apparatus. Optionally, the referencepoint is a position of an orthographic projection of a view zone of aviewer in a vehicle on the imaging plane. Optionally, dx can beexpressed as a real distance representing a size of a pixel along thefirst direction (e.g., the row direction) of the display apparatus; anddy stands for a real distance representing a size of a pixel along thesecond direction (e.g., the column direction) of the display apparatus.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized in at least one frame from a first frameof image to an (N−1)-th frame of image; and the non-occluded portion ofthe at least partially occluded object is unrecognized in an N-th frameof image, N≥2. Optionally, the computer-readable instructions beingexecutable by a processor to cause the processor to perform determiningwhether the at least partially occluded object is outside field of viewor becomes a fully occluded object in the N-th frame of image; andgenerating predicted second coordinates of the fully occluded object inthe display apparatus corresponding to the N-th frame of image upon adetermination that the at least partially occluded object is stillinside field of view.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized in at least one frame from a first frameof image to an (N−1)-th frame of image; and the non-occluded portion ofthe at least partially occluded object is unrecognized in an N-th frameof image, N≥2. Optionally, the computer-readable instructions beingexecutable by a processor to cause the processor to perform generatingpredicted second coordinates of the at least partially occluded objectin the display apparatus corresponding to the N-th frame of image; anddetermining whether the at least partially occluded object is outsidefield of view based on the predicted second coordinates.

In some embodiments, the non-occluded portion of the at least partiallyoccluded object is recognized from an i-th frame of image to a j-thframe of image, i<j<N; and the non-occluded portion of the at leastpartially occluded object is unrecognized from a (j+1)-th frame of imageto an N-th frame of image. Optionally, the computer-readableinstructions being executable by a processor to cause the processor toperform generating predicted second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image according to Equation (5) and Equation (6):

$\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$

wherein (u_(n), v_(n)) is the second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image; (u_(i), v_(i)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the j-th frame of image; m₁=j−i+1; and m₂=n−j.

The foregoing description of the embodiments of the invention has beenpresented for purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formor to exemplary embodiments disclosed. Accordingly, the foregoingdescription should be regarded as illustrative rather than restrictive.Obviously, many modifications and variations will be apparent topractitioners skilled in this art. The embodiments are chosen anddescribed in order to explain the principles of the invention and itsbest mode practical application, thereby to enable persons skilled inthe art to understand the invention for various embodiments and withvarious modifications as are suited to the particular use orimplementation contemplated. It is intended that the scope of theinvention be defined by the claims appended hereto and their equivalentsin which all terms are meant in their broadest reasonable sense unlessotherwise indicated. Therefore, the term “the invention”, “the presentinvention” or the like does not necessarily limit the claim scope to aspecific embodiment, and the reference to exemplary embodiments of theinvention does not imply a limitation on the invention, and no suchlimitation is to be inferred. The invention is limited only by thespirit and scope of the appended claims. Moreover, these claims mayrefer to use “first”, “second”, etc. following with noun or element.Such terms should be understood as a nomenclature and should not beconstrued as giving the limitation on the number of the elementsmodified by such nomenclature unless specific number has been given. Anyadvantages and benefits described may not apply to all embodiments ofthe invention. It should be appreciated that variations may be made inthe embodiments described by persons skilled in the art withoutdeparting from the scope of the present invention as defined by thefollowing claims. Moreover, no element and component in the presentdisclosure is intended to be dedicated to the public regardless ofwhether the element or component is explicitly recited in the followingclaims.

1. A method for tracking an at, least partially occluded object.comprising: recognizing a non-occluded portion of the at least partiallyoccluded object in an input image; generating a simulated image of theat least partially occluded object based on features of the non-occludedportion extracted from the input image; determining first coordinates ofthe at least partially occluded object in a first coordinate system; andconverting the first coordinates of the at least partially occludedobject in the first coordinate system into second coordinates in asecond coordinate system defined in a display apparatus.
 2. The methodof claim 1, wherein generating the simulated image of the at leastpartially occluded object comprises classifying the non-occluded portionusing classifiers trained using extracted features of portions of aplurality of training objects; wherein the simulated image is an imageof one of the plurality of training objects, a portion of whichcomprising extracted features closest matching the features of thenon-occluded portion extracted from the input image.
 3. The method ofclaim 1, wherein converting the first coordinates of the at leastpartially occluded object in the first coordinate system into the secondcoordinates in the second coordinate system defined in the displayapparatus comprises: converting the first coordinates of the at leastpartially occluded object in the first coordinate system into thirdcoordinates in a third coordinate system; and converting the thirdcoordinates in the third coordinate system into the second coordinatesin the second coordinate system defined in the display apparatus;wherein the first coordinate system is a three-dimensional coordinatesystem; and the third coordinate system is a two-dimensional coordinatesystem on a plane substantially parallel to an imaging plane of thedisplay apparatus.
 4. The method of claim 3, wherein converting thefirst coordinates of the at least partially occluded object in the firstcoordinate system into the third coordinates in the third coordinatesystem is performed according to Equation (1) and Equation (2) when theimaging plane of the display apparatus is substantially parallel to acoordinate plane of the first coordinate system: $\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$ wherein (x2, y2) stands for the third coordinates; (x1,y1) stands for coordinate differences between the first coordinates anda reference coordinate of a reference point in the first coordinatesystem; f stands for a length of an orthographic projection of adistance between the imaging plane and the reference point on acoordinate axis in the first coordinate system perpendicular to theimaging plane; and l stands for a length of an orthographic projectionof a distance between the at least partially occluded object and thereference point on the coordinate axis in the first coordinate systemperpendicular to the imaging plane.
 5. The method of claim 3, whereinconverting the third coordinates in the third coordinate system into thesecond coordinates in the second coordinate system defined in thedisplay apparatus is performed according to Equation (3) and Equation(4): $\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$ wherein (x2, y2) stands for the third coordinates; (u, v)stands for the second coordinates; (u₀, v₀) stands for a referencecoordinate of a reference point in the third coordinate system; dxstands for a real distance representing a first pixel pitch along afirst direction of the display apparatus; dy stands for a real distancerepresenting a second pixel pitch along a second direction of thedisplay apparatus.
 6. The method of claim 1, wherein the non-occludedportion of the at least partially occluded object is recognized in atleast one frame from a first frame of image to an (N−1)-th frame ofimage; and the non-occluded portion of the at least partially occludedobject is unrecognized an N-th frame of image, N≥2; the method furthercomprises: determining whether the at least partially occluded object isoutside field of view or becomes a fully occluded object in the N-thframe of image; generating predicted second coordinates of the fullyoccluded object in the display apparatus corresponding to the N-th frameof image upon a determination that the at least partially occludedobject is still inside field of view; and displaying the simulated imageat a position having the predicted second coordinates in the displayapparatus.
 7. The method of claim 1, wherein the non-occluded portion ofthe at least partially occluded object is recognized in at least oneframe of image from a first frame of image to an (N−1)-th frame ofimage; and the non-occluded portion of the at least partially occludedobject is unrecognized in an N-th frame of image, N≥2; the methodfurther comprises: generating predicted second coordinates of the atleast partially occluded object in the display apparatus correspondingto the N-th frame of image; determining whether the at least partiallyoccluded object is outside field of view based on the predicted secondcoordinates; displaying the simulated image at a position having thepredicted second coordinates in the display apparatus upon adetermination that the at least partially occluded object is stillinside field of view but becomes unrecognizable or fully occluded. 8.The method of claim 1, wherein the non-occluded portion of the at leastpartially occluded object is recognized from an i-th. frame of image toa j-th frame of image, i<j<N; and the non-occluded portion of the atleast partially occluded object is unrecognized from a (j+1)-th frame ofimage to an N-th frame of image, N≥4; the method further comprises:generating predicted second coordinates of the at least partiallyoccluded object in the display apparatus corresponding to the N-th frameof image according to Equation (5) and Equation (6): $\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$ wherein (u_(n), v_(n)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the N-th frame of image; (u_(i), v_(i)) is the second coordinates ofthe at least partially occluded object in the display apparatuscorresponding to the j-th frame of image; m₁=j−i+1; and m₂=n−j.
 9. Themethod of claim 1, further comprising displaying the simulated image ata position having the second coordinates in the display apparatus. 10.An apparatus for tracking an at least partially occluded object,comprising: a memory; and one or more processors; wherein the memory andthe one or more processors are connected with each other; and the memorystores computer-executable instructions for controlling the one or moreprocessors to: recognize a non-occluded portion of the at leastpartially occluded object in an input image; generate a simulated imageof the at least partially occluded object based on features of thenon-occluded portion extracted from the input image; determine firstcoordinates of the at least partially occluded object in a firstcoordinate system; and convert the first coordinates of the at leastpartially occluded object in the first coordinate system into secondcoordinates in a second coordinate system defined in a displayapparatus.
 11. The apparatus of claim 10, wherein the memory furtherstores computer-executable instructions for controlling the one or moreprocessors to classify the non-occluded portion using classifierstrained using extracted features of portions of a plurality of trainingobjects; wherein the simulated image is an image of one of the pluralityof training objects, a portion of which comprising extracted featuresclosest matching the features of the non-occluded portion extracted fromthe input image.
 12. The apparatus of claim 10, wherein the memoryfurther stores computer-executable instructions for controlling the oneor more processors to: convert the first coordinates of the at leastpartially occluded object in the first coordinate system into thirdcoordinates in a third coordinate system; and convert the thirdcoordinates in the third coordinate system into the second coordinatesin the second coordinate system defined in the display apparatus;wherein the first coordinate system is a three-dimensional coordinatesystem; and the third coordinate system is a two-dimensional coordinatesystem on a plane substantially parallel to an imaging plane of thedisplay apparatus.
 13. The apparatus of claim 12, wherein the memoryfurther stores computer-executable instructions for controlling the oneor more processors to convert the first coordinates of the at leastpartially occluded object in the first coordinate system into the thirdcoordinates in the third coordinate system according to Equation (1) andEquation (2) when the imaging plane of the display apparatus issubstantially parallel to a coordinate plane of the first coordinatesystem: $\begin{matrix}{{x_{2} = {x_{1}\frac{f}{1}}};} & (1) \\{{y_{2} = {y_{1}\frac{f}{1}}};} & (2)\end{matrix}$ wherein (x2, y2) stands for the third coordinates; (x1,y1) stands for coordinate differences between the first coordinates anda reference coordinate of a reference point in the first coordinatesystem; f stands for a length of an orthographic projection of adistance between the imaging plane and the reference point on acoordinate axis in the first coordinate system perpendicular to theimaging plane; and l stands for a length of an orthographic projectionof a distance between the at least partially occluded object and thereference point on the coordinate axis in the first coordinate systemperpendicular to the imaging plane.
 14. The apparatus of claim 12,wherein the memory further stores computer-executable instructions forcontrolling the one or more processors to convert the third coordinatesin the third coordinate system into the second coordinates in the secondcoordinate system defined in the display apparatus according to Equation(3) and Equation (4): $\begin{matrix}{{u = {\frac{x_{2}}{dx} + u_{0}}};} & (3) \\{{v = {\frac{y_{2}}{dy} + v_{0}}};} & (4)\end{matrix}$ wherein (x2, y2) stands for the third coordinates; (u, v)stands for the second coordinates; (u₀, v₀) stands for a referencecoordinate of a reference point in the third coordinate system; dxstands for a real distance representing a first pixel pitch along afirst direction of the display apparatus; dy stands for a real distancerepresenting a second pixel pitch along a second direction of thedisplay apparatus.
 15. The apparatus of claim 10, wherein thenon-occluded portion of the at least partially occluded object isrecognized in at least one frame from a first frame of image to an(N−1)-th frame of image; and the non-occluded portion of the at leastpartially occluded object is unrecognized in an N-th frame of image,N≥2; wherein the memory further stores computer-executable instructionsfor controlling the one or more processors to: determine whether the atleast partially occluded object is outside field of view or becomes afully occluded object in the N-th frame of image; and generate predictedsecond coordinates of the fully occluded object in the display apparatuscorresponding to the N-th frame of image upon a determination that theat least partially occluded object is still inside field of view;wherein the apparatus further comprises a display apparatus configuredto display the simulated image at a position having the predicted secondcoordinates in the display apparatus.
 16. The apparatus of claim 10,wherein the non-occluded portion of the at least partially occludedobject is recognized in at least one frame from a first frame of imageto an (N−1)-th frame of image; and the non-occluded portion of the atleast partially occluded object is unrecognized in an N-th frame ofimage, N≥2; wherein the memory further stores computer-executableinstructions for controlling the one or more processors to: generatepredicted second coordinates of the at least partially occluded objectin the display apparatus corresponding to the N-th frame of image; anddetermine whether the at least partially occluded object is outsidefield of view based on the predicted second coordinates; wherein theapparatus further comprises a display apparatus configured to displaythe simulated image at a position having the predicted secondcoordinates in the display apparatus upon a determination that the atleast partially occluded object is still inside field of view butbecomes unrecognizable or fully occluded.
 17. The apparatus of claim 10,wherein the non-occluded portion of the at least partially occludedobject is recognized from an i-th frame of image to a j-th frame ofimage, i<j<N; and the non-occluded portion of the at least partiallyoccluded object is unrecognized from a (j+1)-th frame of image to anN-th frame of image, N≥4; wherein the memory further storescomputer-executable instructions for controlling the one or moreprocessors to: generate predicted second coordinates of the at leastpartially occluded object in the display apparatus corresponding to theN-th frame of image according to Equation (5) and Equation (6):$\begin{matrix}{{u_{n} = {{\left( {u_{j} - u_{i}} \right)\frac{m_{2}}{m_{1}}} + u_{i}}};} & (5) \\{{v_{n} = {{\left( {v_{j} - v_{i}} \right)\frac{m_{2}}{m_{1}}} + v_{i}}};} & (6)\end{matrix}$ wherein (u_(n), v_(n)) is the second coordinates of the atleast partially occluded object in the display apparatus correspondingto the N-th frame of image; (u_(i), v_(i)) is the second coordinates ofthe at least partially occluded object in the display apparatuscorresponding to the j-th frame of image; m₁=j−i+1; and m₂=n−j.
 18. Theapparatus of claim 10, wherein the apparatus further comprises a displayapparatus configured to display the simulated image at a position havingthe second coordinates in the display apparatus.
 19. A vehicle,comprising the apparatus of claim
 10. 20. A computer-program productcomprising a non-transitory tangible computer-readable medium havingcomputer-readable instructions thereon, the computer-readableinstructions being executable by a processor to cause the processor toperform: recognizing a non-occluded portion of an at least partiallyoccluded object in an input image; generating a simulated image of theat least partially occluded object based on features of the non-occludedportion extracted from the input image; determining first coordinates ofthe at least partially occluded object a first coordinate system; andconverting the first coordinates of the at least partially occludedobject in the first coordinate system into second coordinates in asecond coordinate system defined in a display apparatus.